Improving Model Performance Using Dynamic Evaluation and Proper Objective Function

Models have become important decision making aids. Model evaluation (i.e., global sensitivity analysis, calibration and uncertainty analysis), is crucial to improve their prediction accuracy and reduce the likelihood of making decisions that could lead to undesirable policy outcomes. The conventional approach assumes that model parameters are insensitive to season irrespective of the temporal variability of input forcings such as rainfall. This assumption could significantly compromise model performance for low flow seasons and/or high flow seasons depending on the calibration method pursued. This study will demonstrate the advantage of dynamic (seasonal) model evaluation in improving performance compared to the traditional approach. In addition, the impact of the goodnessof-fit criteria (e.g., mean of sum of square of residuals, Nash-Sutcliffe efficiency criteria, volume based efficiency criteria, etc) used as an objective function during automatic calibration on model performance has been examined. Objective functions that would improve the accuracy of simulating high flows as well as low flows were identified. The added values of using multiobjective calibration, over the more widely used single objective calibration, has also been explored. The Little River Experimental Watershed, one of the U.S. Department of Agriculture’s experimental watersheds, has been used to illustrate the approaches tested in the study. Soil and Water Assessment Tool is the watershed simulation model used for the work. Results show that the season based model calibration approach significantly improved model performance, and calibration is sensitive to the efficiency measure used as object function. As such, multiple efficiency criteria should be used to report model performance as no single efficiency measure performed consistently well in describing goodness of model results. Another important finding is that parameter values that are significantly divergent from their “true” values may lead to model performance that may be considered near perfect even when judged using multiple efficiency measures underlining the challenge of parameter identifiability.